1. Field of the Invention
The present invention relates to communicating between Fiber Channel (FC) or Fiber Connectivity (FICON) protocol networks separated by Metro Area Network (MAN) or other Wide Area network (WAN) using the Generic Framing Procedure (GFP); and, in particular, to using GFP control packets to coordinate the use of distance extension at the separated FC/FICON networks.
2. Description of the Related Art
Networks of general purpose computer systems and special devices connected by external communication links are well known. The networks often include one or more network devices that facilitate the passage of information between the computer systems and special devices. A network node is a network device or computer system or other special device connected by the communication links.
Information is exchanged between network nodes according to one or more of many well known, new or still developing protocols. In this context, a protocol consists of a set of rules defining how the nodes interact with each other based on information sent over the communication links. The protocols are effective at different layers of operation within each node, from generating and receiving physical signals of various types, to selecting a link for transferring those signals, to the format of information indicated by those signals, to identifying which software application executing on a computer system sends or receives the information.
Communications between nodes are typically effected by exchanging discrete packets of data. Each packet typically comprises 1] header information associated with a particular protocol, and 2] payload information that follows the header information and contains information that may be processed independently of that particular protocol. In some protocols, the packet includes 3] trailer information following the payload and indicating the end of the payload information. The header includes information such as the source of the packet, its destination, the length of the payload, and other properties used by the protocol. Often, the data in the payload for the particular protocol includes a header and payload for a different protocol. The header for a particular protocol typically indicates a type for the next protocol contained in its payload. The protocol in the payload is said to be encapsulated in the preceding protocol. The portion of a data packet including a header and payload for a particular protocol is also called a packet, a data packet, a frame, a cell, or a message of that protocol. In some publications, these different words are used to distinguish one portion of a data packet from another (e.g., to distinguish a first protocol message, from a second protocol frame), but herein, the terms are used interchangeably.
Some protocols pass protocol-related information among two or more network nodes in special control packets that are communicated separately and which include a payload of information used by the protocol itself rather than a payload of data to be communicated for another application. These control packets and the processes at network nodes that utilize the control packets are said to be in another dimension, a “control plane,” distinct from the “data plane” dimension that includes the data packets with payloads for other applications. The control plane packets are also called management packets. For example, during establishment of a connection using a protocol, some handshaking procedures at communicating nodes involve sending information back and forth in control plane packets for use by that protocol subsequently when transporting data plane packets.
Optical fiber has emerged as a preferred link medium for exchanging large amounts of data because of its high bandwidth, low power consumption, and low noise. Because of attenuation of light intensity in optical cable, however, early optical networks were confined to providing high bandwidth connections (“wide pipes”) into high volume data storage devices over limited-distances. Such limited-distance optical networks, called herein local-area optical networks, formed a basis for storage area networks (SANs). SANs are heavily used in some businesses, such as financial and banking services, in which data of high value to millions of customers are stored and processed frequently.
Several networking protocols have found use in SANs, including Gigabyte Ethernet, Fiber Channel (FC), Enterprise Systems CONnectivity (ESCON) of IBM™ of Armonk, N.Y., and a version of ESCON called Single-Byte Command Code Sets Connection (SBCON) approved by the American National Standards Institute (ANSI), for the command set used by ESCON, known as FC-SB. An updated protocol for faster data throughput at mainframe computers using the same command set is Fiber Connectivity (FICON). The most common SAN technology is Fiber Channel networking (also called Fibre Channel in some publications) with a Small Computer Systems Interface (SCSI) command set.
A typical FC SAN is made up of a number of FC switches which are connected together to form a fabric or network. FC is described in ANSI X3.230-1994, available from domain webstore.ansi.org on the World Wide Web (WWW). The entire contents of ANSI X3.230-1994 are herby incorporated by reference as if fully set forth herein. FC provides for bi-directional, gigabit-per second, transport in FC data packets (also called “frames”) that have a standardized set of bits. FC links are limited by the standard to no more than 10 kilometers (km, 1 km=103 meters) to ensure packets are received before drastic attenuation within a certain time interval used to detect, correct and prevent data loss.
One fallout of recent man-made and natural disasters is the recognition that such valuable data is best redundantly distributed at widely geographically spaced locations. Optical networks that span such scales include metropolitan area networks (MANs) and wide area networks (WANs). Optical MANs and WANs have been developed using special optical nodes that repeat and switch optical signals, correcting for attenuation and other signal degradation. Optical protocols for these larger distance networks (including both MANs and WANs) include a Synchronous Optical Network (SONET), Synchronous Digital Hierarchy (SDH) and Optical Transport Network (OTN) protocols.
Several of the SAN protocols however, do not mesh directly with these larger distance optical protocols. For example, FC is limited to maximum distances of 10 kilometers, while SONET networks can span hundreds and thousands of kilometers. In a common approach, FC ports connected by FC links are mapped to SONET or SDH paths for transport across a SONET or SDH optical transport network. This approach is commonly implemented by encapsulating FC data frames into data packets of a Generic Framing Procedure (GFP). The GFP frames are then encapsulated into the optical transport protocol (e.g., SONET or SDH) for transport across the optical transport network. GFP is described in International Telecommunication Union (ITU), Telecommunication Standardization Sector, Series G: Transmission Systems and Media, Digital Systems and Networks, Series Y: Global Information Infrastructure and Internet Protocol Aspects, Internet Protocol Aspects—Transport, Generic Framing Procedure (ITU-T G.7041/Y.1303), Geneva, Switzerland, December 2001, the entire contents of which are herby incorporated by reference as if fully set forth herein.
Further refinements have been adopted to make better use of the bandwidth of the optical transport network. The refinement for FC is described in the following paragraphs.
In one mode of operating locally, FC nodes control data flow by requiring a FC control plane acknowledgment (ACK) message be received from the far end node for every FC data packet sent by a near end node. In another local mode used between two linked nodes, a group of data packets can be transmitted before receiving the first control plane acknowledgment. The group size is called a buffer-to-buffer credit (BBC), and the control plane acknowledgement is by a R-Rdy Primitive (READY) signal. For links of 10 km and less, the delay introduced by the round trip of data and ACK message or READY signal is small (on the order of 0.033 milliseconds and less, where 1 millisecond, ms, =10−3 second). For links of thousands of miles, this delay can be substantial (over 3 ms). The procedures and buffer sizes designed for the short haul are wasteful of bandwidth on the longer haul. Bandwidth on the optical transport network between SANs goes unused as FC nodes wait for ACK messages or READY signals using procedures designed for shorter distances.
To alleviate the delay and wasted bandwidth, a distance extension processing mode has been introduced at FC nodes on the border between the local SAN and the long-distance, extended area optical transport network. Such nodes are called FC edge nodes hereinafter. According to the distance extension processing mode, the FC edge node devotes extra storage for FC data frames and sends ACK messages and READY signals for all FC data frames directed through the long-distance optical transport network, while the FC edge nodes have room to store those FC data frames. Thus FC nodes on the local FC network receive those ACK messages and READY signals quickly, and continue to send more FC data frames. As a consequence, there is less waiting in the local FC network, and better use is made of available bandwidth over the long-distance or, at least, non-local optical transport network between remote SANs. The distance extension is describe in more detail by V. Devdas et al., “Method and System for Efficient Flow Control for Client Data Frames over GFP across a SONET/SDH Transport Path,” in United States Patent Application Publication US2005/0002338, Jan. 6, 2005, the entire contents of which are hereby incorporated by references as if fully set forth herein.
A port on an FC edge node is manually configured to employ the distance extension mode or not. Successful use of the distance extension involves ports on both FC edge nodes at the two remote SANs operating in concert. Therefore, it is important that both FC edge node ports be configured with the same distance extension mode, whether on or off (also termed “enabled” or “disabled,” respectively). For example, the BBC might be 32 buffers for FC frames when distance extension is disabled, but substantially larger, say 1024 buffers, when distance extension is enabled. Thus some signaling is useful, whereby the local FC edge node (also called the near FC edge node) notifies the distant or remote FC edge node (called hereinafter the far FC edge node) about the distance extension mode used on the near mode. The original FC Protocol standard, however, did not anticipate this signaling need and does not provide a field in FC frames to specify the distance extension mode used. FC frames are identical for nodes using the distance extension and those not, it is the processing and response to the FC frames that differ.
In several approaches, new values are defined for one or more existing two-byte fields in the GFP header, e.g., the Payload Type ID (PTI) and User Payload ID (UPI). One new value signals DE enabled; and the other signals DE disabled. When the far FC edge node receives the GFP header that indicates the near FC edge node DE mode, the far FC edge node can determine whether the DE mode used by the near FC edge node matches the DE mode used by the corresponding port on the far FC edge node. If the modes do not match, an alert is presented so that a network administrator for the far FC edge node is made aware of the mismatch. Corrective action can then be taken. For example, an FC port is reconfigured, or a connection is made with a different port on the near or far FC edge node that does match.
While suitable in some instances, there are disadvantages with these approaches. One disadvantage arises because the GFP header is typically processed in hardware configured as a GFP chip. That hardware does not recognize the new values defined for some of the fields and ignores GFP frames with those values set. Thus users of these approaches must replace the GFP chip with different hardware, such as a field programmable gate array (FPGA) or new chip from another vendor, or with GFP header software that executes more slowly than hardware. Replacement of multiple GFP chips is costly and can be prohibitive for organizations that have tens to hundreds of FC edge nodes already deployed with the older GFP chips.
Based on the foregoing, there is a clear need for techniques to ensure synchronized processing by remote FC edge nodes separated by an extended area optical transport network that do not suffer one or more disadvantages of prior art approaches. In particular, there is a need to ensure synchronized use of the FC distance extension mode by remote FC edge nodes without replacement of older GFP chips.